1. Field of the Invention
The present invention relates to the field of video stabilization and more particularly to the field of registering frames of video.
2. Description of Related Art
As is known, a video consist of a series of frames taken over time. For example, a sensor can be used that captures an image and the image can be saved as one of a series of frames in some form of memory. By taking a series of frames over time, such as 30 frames per second, a video may be formed that can be watched by a user. In order for the sensor to function, typically the sensor is mounted in a housing and an optics module is used to focus the desired image in the focal plane on the sensor so that the image can be processed and stored. The housing, optics module and the sensor are part of the platform and the overall system is familiar to a person of skill in the art. However, one potential problem is that if the platform is shifted while the image is being captured, the captured imaged will reflect the platform shift and repeated platform movement (platform shake) can introduce undesirable jitter that can make it difficult to watch the resultant video. This issue is common in aircraft mounted platforms and is also prevalent with video captured with portable, handheld video platforms such as camcorders that are inherently problematic from a standpoint of platform shake.
To compensate, some video platforms include a mechanism to estimate and correct for platform shake. One approach is to attempt to compensate for platform shake by attempting to isolate the optics and sensor movement from the housing movement. Vibration damping and gyroscopes can be used to help stabilize the optics and sensor but this approach makes the hardware more expensive. Another approach is to use an oversized image sensor and couple the image being recorded to one or more accelerometers so that movement of the platform can be compensated by appropriately chipping out a shifted portion of the image that is being recorded. As can be appreciated, however, both of these techniques require specialized hardware and once the video is taken it is generally not possible to use these techniques to correct platform shake that is still perceptible.
Another method of compensating for platform shake has been to use software to register consecutive frames to the focal plane. As the software method works on frames that are previously captured, the software method can be used in substantially real-time or as a post-processing step at some later point. Traditionally, however, it has been both computationally expensive and algorithmically difficult to register consecutive frames containing multiple movers. A common approach has been to segment different movers and through a number of steps isolate the background movement and remove the translation associated with the background so as to register consecutive frames. While this works relatively well for one or two movers, this method has proven to be difficult when numerous individual movers are included in the frame along with, for example, changes in shadows and the like. Registration is even more difficult if non-uniform transparencies such as suspended condensed water vapor or suspended particulates (e.g., clouds or smoke) are also present in the frame. Therefore, improved methods of compensating for platform shake (e.g., to remove background displacement between frames) would be beneficial so as to help register consecutive frames of video.